<html>  
    <head>
        <title>Transfer-queue Over Globus (TOG) - Going Global with Globus and Grid Engine</title>
    </head>
           
    <body>  
        <h1>Transfer-queue Over Globus (TOG) - Going Global with Globus and Grid Engine</h1>

        <h2>Introduction</h2>

        <p>
        The Transfer-queue Over Globus (TOG) software allows an
        enterprise to access remote compute resources via their
        existing local <a href=http://gridengine.sunsource.net/>Grid
        Engine</a> installation. The TOG software integrates Grid
        Engine V5.3 and <a href=http://www.globus.org/>Globus Toolkit
        V2.2.4</a> to provide this access to remote resources.  The
        TOG software has been built as part of the EPCC <a
        href=http://www.epcc.ed.ac.uk/sungrid/>Sun Data and Computer
        Grids project</a>.  This project aims to develop an
        industry-strength, fully Globus-enabled compute and data
        scheduler based around Grid Engine, Globus plus a wide variety
        of data technologies. The project started in February 2002 and
        will run until January 2004. The partners are the National
        e-Science Centre, represented in this project by EPCC, and Sun
        Microsystems.
        </p>

        <h2>Background</h2>

        <p>
        Grid Engine allows the efficient use of compute resources
        within an organisation.  However there is some desirable Grid
        functionality that Grid Engine does not provide.
        </p>

        <p>
        Grids can be classified at three different levels.
        </p>
        
        <ul>
            <li>
            Cluster Grid - a single team or project and their
            associated resources.
            </li>

            <li>
            Enterprise Grid - multiple teams and projects but within a
            single organisation, facilitating collaboration of
            resources across the enterprise.
            </li>

            <li>
            Global Grid - linked Enterprise and Cluster Grids,
            providing collaboration amongst organisations.  Grid
            Engine meets the first two levels by allowing a user to
            transparently make use of any number of compute resources
            within an organisation. However, Grid Engine, alone does
            not yet meet the third level.
            </li>

        </ul>

        <p>
        The Globus Toolkit is essentially a Grid API for connecting
        distributed compute and instrument resources.  Integration
        with Globus allows Grid Engine to meet this third level.  That
        is, it allows collaboration amongst enterprises.  The TOG
        software integrates Grid Engine V5.3 and Globus Toolkit V2.2.4
        to provide a user with this access to remote resources.  This
        allows user jobs to be scheduled across a global grid.
        </p>

        <h2>Download</h2>

        <p>
        The <a href=http://gridengine.sunsource.net/download/TOG/tog-howto.pdf>How-To document</a> describes the
        installation of TOG, its usage and a trouble-shooting guide.
        </p>

        <p>
        TOG can be downloaded either as a binary or source archive:
        </p>
        <ul>
          <li>Binary: <a href=http://gridengine.sunsource.net/download/TOG/tog-bin.tar.gz>tog-bin.tar.gz</a></li>
          <li>Source: <a href=http://gridengine.sunsource.net/download/TOG/tog-src.tar.gz>tog-src.tar.gz</a></li>
        </ul>

        <p>
        The binary archive contains the TOG binaries, API
        documentation and the How-To document.
        </p>

        <p>
        The source archive contains the TOG source, system tests that
        integrate with the Grid Engine testsuite, developer documents
        (design and testplan), and the How-To document.
        </p>


        <h2>Projects using TOG</h2>

        <p>
        The TOG software has been used to create a compute grid
        between the universities of Glasgow and Edinburgh in the UK.
        Researchers at the Glasgow site of the National e-Science
        Centre have been able to access compute resources at EPCC
        using a Grid Engine installation configured with the TOG
        software.  TOG is also being used to set up a biomedical
        e-Science demonstration using the new SRIF network linking
        three sites within the University of Edinburgh - EPCC, the
        Scottish Centre for Genomic Technology and Informatics (GTI)
        at the New Royal Infirmary of Edinburgh and the MRC Human
        Genetics Unit (HGU) at the Western General Hospital.
        </p>

        <h2>Future Delevopments</h2>

        <p>
        Following the development of the TOG software, the project has
        investigated the integration of access to data sources via
        data grid technologies such as OGSA-DAI, GridFTP and SRB.  The
        next step for the project team is to develop a hierarchical
        scheduler that scales better in a grid environment and enables
        access to remote data sources via data grid technologies.
        This will use an OGSA-compliant Grid Service interface to Grid
        Engine.  The hierarchical scheduler will query child Grid
        Engine installations at collaborating sites to determine if
        they are able to run a user's job. The scheduler will then
        place the user's job at the site that best matches the
        following criteria.
        </p>
        
        <ol>
           <li>It is capable of running the job.</li>
           <li>It has the lowest load of the available sites.</li>
           <li>It has the best access to the required data sources.</li>
        </ol>

        <p>
        For those user jobs that are not data grid aware, a data
        component will handle the transfer of data between sites.  The
        hierarchical scheduler (JOSH) has been released, more
        information can be found 
        <a href="http://gridengine.sunsource.net/josh.html">here</a>.
        </p>

        <p>

        For more information on the project and its deliverables
        please access the EPCC Sun Data and Compute Grids project <a
        href=http://www.epcc.ed.ac.uk/sungrid/>home page</a> or
        contact the <a href=mailto:sungrid@epcc.ed.ac.uk>SunDCG
        project team</a>.

        </p>

    </body>

</html>

